dir.to.work = "C:\\Users\\Yoav\\Documents\\gdrive.mail.tau.ac.il\\Other computers\\My Laptop\\"
source(paste0(dir.to.work, "work\\resources\\stasExamples\\R\\yba.funcs.R"))
# The data's folder (directory)
dir.to.data = "C:\\Users\\Yoav\\OneDrive - Tel-Aviv University\\Documents\\bigfiles\\"
dir.out <- paste0(dir.to.data, "ageformva\\processed")
In previous research, participants who read behavioral evaluative information about a young man and an old man showed a strong pro-young bias in their automatic evaluation of those targets.
In this image, we see the preference for a positively-portrayed
target over a negatively-portrayed target, in the self-repotr evaluation
(on the left) and in the IAT (on the right). The x-axis is the
behavioral information condition. For example, when x is 10, it means
that the positively-portrayed target was characterized by 13 positive
and 5 negative behaviors, whereas the negatively-portrayed was
characterized by 13 negative and 5 positive behaviors.
In the present experiment, we will
describe the positively-portrayed target with 8 positive and 4 negative
behaviors, and the opposite for the negatively-portrayed target. That’s
a difference about 8 behaviors.
The purpose of the present experiment was to investigate possible causes for the group effect. We will try to estimate:
Overview of the procedure:
Baseline. Part 1: Only names; Part 3: Only names. (age groups are provided only in the name-photo matching task).
Age1. Part 1: Name + age; Part 3: Only names.
*Photo1.** Part 1: Name + age + photos; Part 3: Only names.
Photo2. Part 1: Name + age; Part 3: Names+Photos.
PhotoEnd. Part 1: Only names; Part 3: Names+Photos.
Behavioural information manipulation: We manipulated between participants whether the old man or the young man was the positively portrayed target (there was always one positively-portrayed and one negatively-portrayed target).
Measures:
allds <- read.csv(paste(dir.out, "allds.csv", sep = "\\"))
allok <- read.csv(paste(dir.out, "allok.csv", sep = "\\"))
allok$positive <- ifelse(allok$oldVlnc == "g", "old", ifelse(allok$oldVlnc == "b", "yng", NA))
allok$iat.order2 <- ifelse(allok$yngGd.first & !allok$oldGd.first, "yg.first", ifelse(!allok$yngGd.first & allok$oldGd.first, "og.first", ifelse(allok$yngGd.first & allok$oldGd.first, NA, "error")))
allok$iat.order1 <- ifelse(allok$iat.order2 == "yg.first", ifelse(allok$positive == "yng", "gFirst", ifelse(allok$positive == "old", "bFirst", NA)), ifelse(allok$iat.order2 == "og.first", ifelse(allok$positive == "old", "gFirst", ifelse(allok$positive == "yng", "bFirst", NA)), NA))
allok$condition <- factor(paste(allok$part1Cond, allok$part3Cond, sep = ","))
How many reached the final task (the memory test), out of those who started the study
allds$started <- !is.na(allds$task4)
allds$completed <- !is.na(allds$task10)
my.freq(allds$completed[which(allds$started)])
Of the 1431 participants who started the study, 1093 (76%) completed.
The planned exclusions:
We will exclude participants with poor performance in the IAT (above 10% fast trials, Greenwald et al., 2003).
We will exclude participants with less than 70% accuracy rate in the last 20 trials of the matching task.
Participants who did not respond to all the questions in a specific questionnaire will be excluded from all the analyses that include that questionnaire, but not from other analyses.
IAT performance (0 means ok)
my.freq(allds$SUBEXCL)
38 did not perform the IAT well, but probably some of them did not complete the study, dropping out during the IAT (those with a value of 2). So, let’s see the IAT performance status, only among participants who reached the final task:
my.freq(allds$SUBEXCL[which(allds$completed)])
So, we can say that close to 2% of the participnats (20) performed poorly. This is a bit higher than the usual (which is about 1%).
Accuracy in the matching task:
mysum(allds$match.acc, probs = c(0.05, 0.036))
n M SD SE med 5% 3.6%
1162.000 0.936 0.102 0.002 0.958 0.750 0.667
People were generally accurate, with less than 4% performing too poorly.
We also excluded participants with data that suggested a technical mistake (the conditions were not saved correctly)
Overall, here are the number of exclusions and inclusions:
pts <- list()
pts$n.started <- sum(allds$started)
pts$n.completed <- sum(allds$completed, na.rm = T)
pts$IAT.poor <- sum(allds$SUBEXCL[allds$completed] > 0, na.rm = T)
pts$not.all.evaluations <- sum(allds$n.atts != 12, na.rm = T)
pts$tech.prob <- sum(allds$conds.dup[allds$completed] | !allds$cond.ok[allds$completed], na.rm = T)
pts$match.too.many.errors <- sum(allds$SUBEXCL[allds$completed] > 0, na.rm = T)
pts$total.ok <- nrow(allok)
pts
$n.started
[1] 1431
$n.completed
[1] 1093
$IAT.poor
[1] 20
$not.all.evaluations
[1] 39
$tech.prob
[1] 89
$match.too.many.errors
[1] 20
$total.ok
[1] 925
We planned to collect data from 1000 participants, so the sample is 7.5% smaller than we hoped for.
Age:
mysum(allok$age)
n M SD SE med
925.000 35.806 15.295 0.503 31.000
A plot of the age percentiles:
plot.percentile(allok$age, yTitle = "Age", xTitle = "Percentile")
Probably, the 114 years-old lied.
Gender:
my.freq(allok$gender)
Language. We asked: Which of the following is the most accurate? answers: [‘English is my native language’, ‘English is not my native langauge, but it is my primary language’, ‘English is not my native language, and not my primary language, but I am fluent in English’, ‘I understand some English’ ] coded as 1, 2, 3, and 4 (-999 selected a “decline to answer” button at the bottom of the page)
my.freq(allok$lang)
We did not plan to exclude based on this question, but we could explore in the Extended Results section, whether the results change based on this question.
The IAT preference for the positively-portrayed target over the negatively-portrayed target, as a function of the group affiliation of the positively-portrayed target, and the induction and measurement conditions.
Violin plot:
my.violin(DV = allok$IAT.gb, xFactor = allok$condition, fillFactor = allok$positive) + theme(axis.text.x = element_text(size = 11))
(The rectangle is the median, the circle is the mean, and the error-bars
are for SE). The difference between the two groups (old vs. yng) is the
indication of age-bias. It seems clear only in the name+age,name+photo
condition.
Descriptive Statistics:
mysumBy(IAT.gb ~ part1Cond + part3Cond + positive, dt = allok)
Including the IAT’s block order condition (whether the positively portrayed was paired first with Good words or Bad words in the IAT):
knitr::kable(mysumBy(IAT.gb ~ iat.order1 + part1Cond + part3Cond + positive, dt = allok))
| var | iat.order1 | part1Cond | part3Cond | positive | n | M | SD | SE | med |
|---|---|---|---|---|---|---|---|---|---|
| IAT.gb | bFirst | name-only | name-only | old | 50 | 0.216 | 0.440 | 0.062 | 0.196 |
| IAT.gb | bFirst | name-only | name-only | yng | 48 | 0.186 | 0.355 | 0.051 | 0.190 |
| IAT.gb | bFirst | name-only | name+photo | old | 47 | 0.181 | 0.349 | 0.051 | 0.215 |
| IAT.gb | bFirst | name-only | name+photo | yng | 45 | 0.183 | 0.325 | 0.048 | 0.144 |
| IAT.gb | bFirst | name+age | name-only | old | 52 | 0.038 | 0.362 | 0.050 | 0.004 |
| IAT.gb | bFirst | name+age | name-only | yng | 42 | 0.171 | 0.321 | 0.050 | 0.139 |
| IAT.gb | bFirst | name+age | name+photo | old | 47 | 0.042 | 0.298 | 0.044 | 0.083 |
| IAT.gb | bFirst | name+age | name+photo | yng | 46 | 0.308 | 0.348 | 0.051 | 0.321 |
| IAT.gb | bFirst | name+age+photo | name-only | old | 48 | 0.274 | 0.356 | 0.051 | 0.208 |
| IAT.gb | bFirst | name+age+photo | name-only | yng | 48 | 0.176 | 0.391 | 0.056 | 0.163 |
| IAT.gb | gFirst | name-only | name-only | old | 39 | -0.062 | 0.392 | 0.063 | -0.080 |
| IAT.gb | gFirst | name-only | name-only | yng | 43 | 0.043 | 0.427 | 0.065 | 0.029 |
| IAT.gb | gFirst | name-only | name+photo | old | 40 | -0.109 | 0.320 | 0.051 | -0.127 |
| IAT.gb | gFirst | name-only | name+photo | yng | 37 | 0.092 | 0.357 | 0.059 | 0.103 |
| IAT.gb | gFirst | name+age | name-only | old | 56 | -0.126 | 0.351 | 0.047 | -0.083 |
| IAT.gb | gFirst | name+age | name-only | yng | 48 | -0.056 | 0.350 | 0.051 | -0.067 |
| IAT.gb | gFirst | name+age | name+photo | old | 36 | -0.089 | 0.384 | 0.064 | -0.105 |
| IAT.gb | gFirst | name+age | name+photo | yng | 53 | -0.014 | 0.396 | 0.054 | 0.012 |
| IAT.gb | gFirst | name+age+photo | name-only | old | 51 | 0.028 | 0.458 | 0.064 | -0.095 |
| IAT.gb | gFirst | name+age+photo | name-only | yng | 49 | 0.049 | 0.416 | 0.059 | 0.038 |
There was a clear “reverse” block-order effect. That occurs when the expected score (preference for good) is stronger when that the compatible pairing (in our case, good person + good words) comes at the end, rather than at the beginning of the IAT. The hypothesized reason for such an effect is that the practicing on the sides change of the target categories (in Block 5) is too lonq.
When the negatively-portrayed target individual was paired with Good words in Blocks 3 & 4 of the IAT (i.e., first), there was a pro-young bias only in the conditions [name+age, name-only] & (especially in) [name+age, name+photo]. When the positively-portrayed target individual was paired with Bad words in Blocks 3 & 4 of the IAT (i.e., first), the larger pro-young bias was in the condition [name-only, name+photo]. This inconsistency is not very reassuring.
Let’s see it by the complete condition:
knitr::kable(mysumBy(IAT.gb ~ iat.order1 + positive + condition, dt = allok))
| var | iat.order1 | positive | condition | n | M | SD | SE | med |
|---|---|---|---|---|---|---|---|---|
| IAT.gb | bFirst | old | name-only,name-only | 50 | 0.216 | 0.440 | 0.062 | 0.196 |
| IAT.gb | bFirst | old | name-only,name+photo | 47 | 0.181 | 0.349 | 0.051 | 0.215 |
| IAT.gb | bFirst | old | name+age,name-only | 52 | 0.038 | 0.362 | 0.050 | 0.004 |
| IAT.gb | bFirst | old | name+age,name+photo | 47 | 0.042 | 0.298 | 0.044 | 0.083 |
| IAT.gb | bFirst | old | name+age+photo,name-only | 48 | 0.274 | 0.356 | 0.051 | 0.208 |
| IAT.gb | bFirst | yng | name-only,name-only | 48 | 0.186 | 0.355 | 0.051 | 0.190 |
| IAT.gb | bFirst | yng | name-only,name+photo | 45 | 0.183 | 0.325 | 0.048 | 0.144 |
| IAT.gb | bFirst | yng | name+age,name-only | 42 | 0.171 | 0.321 | 0.050 | 0.139 |
| IAT.gb | bFirst | yng | name+age,name+photo | 46 | 0.308 | 0.348 | 0.051 | 0.321 |
| IAT.gb | bFirst | yng | name+age+photo,name-only | 48 | 0.176 | 0.391 | 0.056 | 0.163 |
| IAT.gb | gFirst | old | name-only,name-only | 39 | -0.062 | 0.392 | 0.063 | -0.080 |
| IAT.gb | gFirst | old | name-only,name+photo | 40 | -0.109 | 0.320 | 0.051 | -0.127 |
| IAT.gb | gFirst | old | name+age,name-only | 56 | -0.126 | 0.351 | 0.047 | -0.083 |
| IAT.gb | gFirst | old | name+age,name+photo | 36 | -0.089 | 0.384 | 0.064 | -0.105 |
| IAT.gb | gFirst | old | name+age+photo,name-only | 51 | 0.028 | 0.458 | 0.064 | -0.095 |
| IAT.gb | gFirst | yng | name-only,name-only | 43 | 0.043 | 0.427 | 0.065 | 0.029 |
| IAT.gb | gFirst | yng | name-only,name+photo | 37 | 0.092 | 0.357 | 0.059 | 0.103 |
| IAT.gb | gFirst | yng | name+age,name-only | 48 | -0.056 | 0.350 | 0.051 | -0.067 |
| IAT.gb | gFirst | yng | name+age,name+photo | 53 | -0.014 | 0.396 | 0.054 | 0.012 |
| IAT.gb | gFirst | yng | name+age+photo,name-only | 49 | 0.049 | 0.416 | 0.059 | 0.038 |
Let’s try to see which conditions show a pro-young bias, by looking at the pro-positive IAT score (preference for the positively-portrayed), by IAT-order condition and by positive-target’s group condition.
knitr::kable(mysumBy(IAT.gb ~ condition + iat.order1 + positive, dt = allok))
| var | condition | iat.order1 | positive | n | M | SD | SE | med |
|---|---|---|---|---|---|---|---|---|
| IAT.gb | name-only,name-only | bFirst | old | 50 | 0.216 | 0.440 | 0.062 | 0.196 |
| IAT.gb | name-only,name-only | bFirst | yng | 48 | 0.186 | 0.355 | 0.051 | 0.190 |
| IAT.gb | name-only,name-only | gFirst | old | 39 | -0.062 | 0.392 | 0.063 | -0.080 |
| IAT.gb | name-only,name-only | gFirst | yng | 43 | 0.043 | 0.427 | 0.065 | 0.029 |
| IAT.gb | name-only,name+photo | bFirst | old | 47 | 0.181 | 0.349 | 0.051 | 0.215 |
| IAT.gb | name-only,name+photo | bFirst | yng | 45 | 0.183 | 0.325 | 0.048 | 0.144 |
| IAT.gb | name-only,name+photo | gFirst | old | 40 | -0.109 | 0.320 | 0.051 | -0.127 |
| IAT.gb | name-only,name+photo | gFirst | yng | 37 | 0.092 | 0.357 | 0.059 | 0.103 |
| IAT.gb | name+age,name-only | bFirst | old | 52 | 0.038 | 0.362 | 0.050 | 0.004 |
| IAT.gb | name+age,name-only | bFirst | yng | 42 | 0.171 | 0.321 | 0.050 | 0.139 |
| IAT.gb | name+age,name-only | gFirst | old | 56 | -0.126 | 0.351 | 0.047 | -0.083 |
| IAT.gb | name+age,name-only | gFirst | yng | 48 | -0.056 | 0.350 | 0.051 | -0.067 |
| IAT.gb | name+age,name+photo | bFirst | old | 47 | 0.042 | 0.298 | 0.044 | 0.083 |
| IAT.gb | name+age,name+photo | bFirst | yng | 46 | 0.308 | 0.348 | 0.051 | 0.321 |
| IAT.gb | name+age,name+photo | gFirst | old | 36 | -0.089 | 0.384 | 0.064 | -0.105 |
| IAT.gb | name+age,name+photo | gFirst | yng | 53 | -0.014 | 0.396 | 0.054 | 0.012 |
| IAT.gb | name+age+photo,name-only | bFirst | old | 48 | 0.274 | 0.356 | 0.051 | 0.208 |
| IAT.gb | name+age+photo,name-only | bFirst | yng | 48 | 0.176 | 0.391 | 0.056 | 0.163 |
| IAT.gb | name+age+photo,name-only | gFirst | old | 51 | 0.028 | 0.458 | 0.064 | -0.095 |
| IAT.gb | name+age+photo,name-only | gFirst | yng | 49 | 0.049 | 0.416 | 0.059 | 0.038 |
A complete ANOVA:
library(afex)
aov_ez(id = "session_id", dv = "IAT.gb", data = allok, between = c("iat.order1", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))
Only a small effect of bias:
mysumBy(IAT.gb ~ positive, dt = allok)
This effect looks smaller than in previous research (e.g., a beta of .69 in the original study). Notably, the previous research was a [photo-only, photo-only] condition, and without the matching task. Any of these modifications might have reduced the effect of the target’s group affiliation.
The effect of age, within each condition:
aez <- aov_ez(id = "session_id", dv = "IAT.gb", data = allok, between = c("iat.order1", "condition", "positive"))
emm_s.t <- emmeans(aez, pairwise ~ positive | condition)
emm_s.t[[2]]
condition = name-only,name-only:
contrast estimate SE df t.ratio p.value
old - yng -0.0372 0.0559 905 -0.666 0.5056
condition = name-only,name+photo:
contrast estimate SE df t.ratio p.value
old - yng -0.1015 0.0577 905 -1.760 0.0787
condition = name+age,name-only:
contrast estimate SE df t.ratio p.value
old - yng -0.1017 0.0533 905 -1.907 0.0569
condition = name+age,name+photo:
contrast estimate SE df t.ratio p.value
old - yng -0.1708 0.0559 905 -3.056 0.0023
condition = name+age+photo,name-only:
contrast estimate SE df t.ratio p.value
old - yng 0.0385 0.0533 905 0.721 0.4711
Results are averaged over the levels of: iat.order1
The effect was significant only in the name+age,name+photo condition. This might be the most similar to the original method (photo-only, photo-only). It might suggest that it is important to know the group affiliation in the encoding. However, we can’t easily say that there would not be an effect without that, because the [name-only,name+photo] condition might show an effect with a larger sample. Let’s have a look at effects (the estimates) and our main questions. Here are our research questions regarding the IAT:
Is there an age bias in the baseline condition? Definitely not. It is not enough to learn about the age after the encoding, and without showing the age in the IAT.
A comparison of the group effect between Photo1 and Age1 will estimate the effect of visual cues on encoding. name+age+photo,name-only (Photo1) vs. name+age,name-only [Age1] = Well, Photo1 is the only that did not show the bias numerically (a positive estimate), so we will not see evidence here that showing photos during encoding was important. If anything, we will see evidence of the opposite, which would not be easily explained.
A comparison of Photo2 to Age1 will estimate the effect of visual cues on evaluation. name+age,name+photo [Photo2] vs. name+age,name-only [Age1]: The effect was numerically larger in Photo2 than Age1, but probably not significantly so.
A comparison of Age1 to baseline will estimate the effect of mere group knowledge on encoding. name+age,name-only [Age1] vs. name-only,name-only [baseline] = Numerically, it seems that knowing about the age during encoding increased bias, but it is obvious that this difference is not significant.
I haven’t succeeded so far in testing the planned contrasts within the complete ANOVA model. Therefore, for now, we’ll test them with ANOVAs on each pair of conditions (as planned in the plan document). This is a less powerful test (the MSEis likely to be larger), and it doesn’t allow for easy correction of the p-values.
A comparison of Photo2 to Age1 will estimate the effect of visual cues on evaluation. name+age,name+photo [Photo2] vs. name+age,name-only [Age1]
ccc <- allok[which(allok$condition %in% c("name+age,name+photo", "name+age,name-only")), ]
mysumBy(IAT.gb ~ condition + positive, dt = ccc)
aov_ez(id = "session_id", dv = "IAT.gb", data = ccc, between = c("iat.order1", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))
No interaction between condition and the group affiliation of the positively-portrayed target individual on the IAT preference for the positively-portrayed individual over the negatively-portrayed individual. With such a small effect, it is unlikely that we will see a significant effect with any alternative analysis.
A comparison of Age1 to baseline will estimate the effect of mere group knowledge on encoding. name+age,name-only [Age1] vs. name-only,name-only [baseline]
ccc <- allok[which(allok$condition %in% c("name-only,name-only", "name+age,name-only")), ]
mysumBy(IAT.gb ~ condition + positive, dt = ccc)
aov_ez(id = "session_id", dv = "IAT.gb", data = ccc, between = c("iat.order1", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))
No interaction between condition and the group affiliation of the positively-portrayed target individual on the IAT preference for the positively-portrayed individual over the negatively-portrayed. Again, the partial eta square seems too small to amount to a significant difference in any alternative statistical analysis.
A comparison of the group effect between Photo1 and Age1 will estimate the effect of visual cues on encoding. name+age+photo,name-only (Photo1) vs. name+age,name-only [Age1]
ccc <- allok[which(allok$condition %in% c("name+age+photo,name-only", "name+age,name-only")), ]
mysumBy(IAT.gb ~ condition + positive, dt = ccc)
aov_ez(id = "session_id", dv = "IAT.gb", data = ccc, between = c("iat.order1", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))
No interaction between condition and the group affiliation of the positively-portrayed target individual on the IAT preference for the positively-portrayed individual over the negatively-portrayed. This time, perhaps a more powerful statistical analysis would result with a significant difference, although the effect is still very small. Importantly, the difference is odd (stronger bias with the photos of the target individuals), which further reduces the confidence in these results.
Regarding each of the two target individuals, participants responded to 6 questions in the following format: Does the word ATTRIBUTE characterize NAME? In name+photo conditions, the photo of the individual was shown with each question. I will explore these questions further in the “Extended Results” section, but now let’s dive in to the preference score computed from these evaluation questions. We will look at the preference for the positively-portrayed target individual.
my.violin(DV = allok$eval.gb, xFactor = allok$condition, fillFactor = allok$positive) + theme(axis.text.x = element_text(size = 11))
mysumBy(eval.gb ~ part1Cond + part3Cond + positive, dt = allok)
There was always a preference for the positively-portrayed target. In two of the conditions, there seems to be a pro-young bias: in the baseline condition [name-only, name-only] and in the Photo1 [name+age+photo, name-only] conditions. That is unexpected.
Let’s ANOVA in a 5 (condition) x 2 (positive target) ANOVA
aov_ez(id = "session_id", dv = "eval.gb", data = allok, between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
Well, there is no interaction, only a tiny effect of the group of the positively-portray target individual.
mysumBy(eval.gb ~ positive, dt = allok)
A slight pro-young preference.
Still, let’s see the effect of the targets’ group within each condition:
aez <- aov_ez(id = "session_id", dv = "eval.gb", data = allok, between = c("condition", "positive"))
emm_s.t <- emmeans(aez, pairwise ~ positive | condition)
emm_s.t[[2]]
condition = name-only,name-only:
contrast estimate SE df t.ratio p.value
old - yng -16.626 7.01 915 -2.372 0.0179
condition = name-only,name+photo:
contrast estimate SE df t.ratio p.value
old - yng 2.606 7.24 915 0.360 0.7188
condition = name+age,name-only:
contrast estimate SE df t.ratio p.value
old - yng -3.336 6.71 915 -0.497 0.6192
condition = name+age,name+photo:
contrast estimate SE df t.ratio p.value
old - yng -0.819 7.00 915 -0.117 0.9069
condition = name+age+photo,name-only:
contrast estimate SE df t.ratio p.value
old - yng -20.352 6.72 915 -3.031 0.0025
Let’s also see those effect in a t-test, mostly to see the Cohen’s d and Bayes Factor: In name+age+photo,name-only
ccc <- allok[which(allok$condition == "name+age+photo,name-only"), ]
ttestIS(data = ccc, vars = c("eval.gb"), group = "positive", bf = T, desc = T, effectSize = T)
INDEPENDENT SAMPLES T-TEST
Independent Samples T-Test
───────────────────────────────────────────────────────────────────────────────────────────────────────────────
Statistic error % df p Effect Size
───────────────────────────────────────────────────────────────────────────────────────────────────────────────
eval.gb Student's t -3.431114 194.0000 0.0007345 Cohen's d -0.4901846
Bayes factor₁₀ 34.17799 7.992516e-10
───────────────────────────────────────────────────────────────────────────────────────────────────────────────
Note. Hₐ μ <sub>old</sub> ≠ μ <sub>yng</sub>
Group Descriptives
──────────────────────────────────────────────────────────────────────────
Group N Mean Median SD SE
──────────────────────────────────────────────────────────────────────────
eval.gb old 99 21.74074 10.66667 37.50179 3.769072
yng 97 42.09278 33.33333 45.25380 4.594828
──────────────────────────────────────────────────────────────────────────
In name-only,name-only (the baseline condition)
ccc <- allok[which(allok$condition == "name-only,name-only"), ]
ttestIS(data = ccc, vars = c("eval.gb"), group = "positive", bf = T, desc = T, effectSize = T)
INDEPENDENT SAMPLES T-TEST
Independent Samples T-Test
──────────────────────────────────────────────────────────────────────────────────────────────────────────────
Statistic error % df p Effect Size
──────────────────────────────────────────────────────────────────────────────────────────────────────────────
eval.gb Student's t -2.221616 178.0000 0.0275698 Cohen's d -0.3311995
Bayes factor₁₀ 1.575094 1.184092e-4
──────────────────────────────────────────────────────────────────────────────────────────────────────────────
Note. Hₐ μ <sub>old</sub> ≠ μ <sub>yng</sub>
Group Descriptives
───────────────────────────────────────────────────────────────────────────
Group N Mean Median SD SE
───────────────────────────────────────────────────────────────────────────
eval.gb old 89 15.02996 0.6666667 51.19278 5.426424
yng 91 31.65568 17.00000 49.20688 5.158283
───────────────────────────────────────────────────────────────────────────
So, two of the conditions showed small-to-medium pro-young bias in the self-reported evaluation. What’s odd is that these conditions did not show that pro-young bias in the IAT. This might suggest that the lack of pro-young IAT bias in the name-only IATs reflect worst measurement rather than less bias. That is, perhaps IATs with names only are less reliable/valid in attitude formation studies (because the name was just learned). We could test that speculation by looking at the effect of condition on internal consistency. We might also learn something by looking at correlations between IAT and self-report and at discrepancies, as a function of the condition.
Overall correlation between the evaluation scores. The IAT.prf and eval.diff are preferences for the young target over the old target.
my.htmlTable(cornp(allok[, c("IAT.gb", "eval.gb", "age", "IAT.prf", "eval.diff")]))
| varName____ | IAT.gb____ | eval.gb____ | age____ | IAT.prf____ | |
|---|---|---|---|---|---|
| 1 | eval.gb |
0.171 < .001 925 |
|||
| 2 | age |
-0.013 0.698 925 |
-0.036 0.273 925 |
||
| 3 | IAT.prf |
-0.021 0.52 925 |
-0.039 0.237 925 |
0.007 0.831 925 |
|
| 4 | eval.diff |
-0.008 0.816 925 |
0.001 0.974 925 |
0.021 0.523 925 |
0.237 < .001 925 |
It might seem odd that the IAT.prf/eval.diff correlation is not the same as the IAT.gb/eval.gb correlation, but I verified (in the sanity checks below) that there is no error here. My guess is that that it can happen if the SDs are changed due to the recoding. In any case, the correlations here are not large, but are in the range of the typical IAT/self-report correlation which is about 0.15-0.70, depending on the attitude object.
Let’s see whether the correlation changed drastically between conditions:
my.htmlTable(corby(inData = allok, xs = c("IAT.gb", "IAT.prf"), ys = c("eval.gb", "eval.diff"), by = c("condition")))
| condition____ | varName____ | eval.gb____ | eval.diff____ | |
|---|---|---|---|---|
| 1 | name-only,name-only | IAT.gb |
0.19 0.011 180 |
-0.121 0.106 180 |
| 2 | name-only,name-only | IAT.prf |
-0.1 0.182 180 |
0.27 < .001 180 |
| 3 | name-only,name+photo | IAT.gb |
0.227 0.003 169 |
0.087 0.261 169 |
| 4 | name-only,name+photo | IAT.prf |
0.024 0.753 169 |
0.316 < .001 169 |
| 5 | name+age,name-only | IAT.gb |
0.004 0.957 198 |
-0.027 0.708 198 |
| 6 | name+age,name-only | IAT.prf |
-0.109 0.125 198 |
0.001 0.989 198 |
| 7 | name+age,name+photo | IAT.gb |
0.202 0.006 182 |
0.063 0.396 182 |
| 8 | name+age,name+photo | IAT.prf |
-0.03 0.691 182 |
0.252 < .001 182 |
| 9 | name+age+photo,name-only | IAT.gb |
0.216 0.002 196 |
-0.055 0.446 196 |
| 10 | name+age+photo,name-only | IAT.prf |
0.037 0.602 196 |
0.358 < .001 196 |
The correlation were relatively consistent, excluding the condition ‘name+age,name-only’ which showed zero correlation. I don’t see anything unordinary in that condition, in terms of discrepancy.
Let’s see effect sizes of the pro-young bias, by condition and measure.
effect_sizes <- by(allok, allok$condition, function(x) {
d_IAT <- cohen.d(x$IAT.gb[x$positive == "yng"], x$IAT.gb[x$positive == "old"])
d_eval <- cohen.d(x$eval.gb[x$positive == "yng"], x$eval.gb[x$positive == "old"])
m_IAT.yng <- mean(x$IAT.gb[x$positive == "yng"], na.rm = T)
m_IAT.old <- mean(x$IAT.gb[x$positive == "old"], na.rm = T)
m_eval.yng <- mean(x$eval.gb[x$positive == "yng"], na.rm = T)
m_eval.old <- mean(x$eval.gb[x$positive == "old"], na.rm = T)
c(IAT.d = d_IAT$estimate, IAT.ci = d_IAT$conf.int, eval.d = d_eval$estimate, eval.ci = d_eval$conf.int, IAT.m.yng = m_IAT.yng, IAT.m.old = m_IAT.old, eval.m.yng = m_eval.yng, eval.m.old = m_eval.old)
})
effect_sizes_df <- as.data.frame(round(do.call(rbind, effect_sizes), 2))
effect_sizes_df$IAT.ci <- paste0("[", effect_sizes_df$IAT.ci.lower, " - ", effect_sizes_df$IAT.ci.upper, "]")
effect_sizes_df$eval.ci <- paste0("[", effect_sizes_df$eval.ci.lower, " - ", effect_sizes_df$eval.ci.upper, "]")
effect_sizes_df$IAT.ci.lower <- NULL
effect_sizes_df$IAT.ci.upper <- NULL
effect_sizes_df$eval.ci.lower <- NULL
effect_sizes_df$eval.ci.upper <- NULL
effect_sizes_df[, c("IAT.d", "IAT.ci", "eval.d", "eval.ci", "IAT.m.yng", "IAT.m.old", "eval.m.yng", "eval.m.old")]
There is an overlap in the CI of the effect-size of the self-report (eval) and the IAT score in all the conditions. The only possible exception is Photo1 [name+age+photo,name-only], with the largest pro-young bias in the self-reported evaluation and the smallest (indeed, pro-old) bias in the IAT. Because this is unexpected and not easily explained, I suspect this is a statistical fluke, until replication.
The memory question was: Who appeared with the following
behavior?
<%=questionsData.bv%>
Response options were either the names or the photos of the two targets, and “None of them”.
We presented 4 positive and 4 negative behaviors of each target, and 4 positive and 4 negative novel behaviors (total of 24 behaviors).
Memory accuracy by type of question (who = who was really characterized by this question [y = young, o = old, n = none], vlnc = whether the behavior was positive or negative [p or n]):
mmm <- mymelt(dt = allok, formula = macc.o.neg + macc.o.pos + macc.y.neg + macc.y.pos + macc.n.neg + macc.n.pos ~ session_id + condition + positive)
mmm$who <- substr(mmm$variable, 6, 6)
mmm$vlnc <- substr(mmm$variable, 8, 8)
mysumBy(value ~ who + vlnc, dt = mmm)
Negative behaviors, behaviors performed by the young target, and behaviors that appeared in the induction were remembered better.
With the positively-portrayed group affiliation condition:
knitr::kable(mysumBy(value ~ vlnc + positive + who, dt = mmm))
| var | vlnc | positive | who | n | M | SD | SE | med |
|---|---|---|---|---|---|---|---|---|
| value | n | old | n | 451 | 0.729 | 0.356 | 0.016 | 1.00 |
| value | n | old | o | 451 | 0.774 | 0.289 | 0.013 | 1.00 |
| value | n | old | y | 451 | 0.835 | 0.246 | 0.011 | 1.00 |
| value | n | yng | n | 454 | 0.743 | 0.343 | 0.016 | 1.00 |
| value | n | yng | o | 454 | 0.792 | 0.262 | 0.012 | 1.00 |
| value | n | yng | y | 454 | 0.849 | 0.255 | 0.012 | 1.00 |
| value | p | old | n | 451 | 0.684 | 0.358 | 0.017 | 0.75 |
| value | p | old | o | 451 | 0.713 | 0.280 | 0.013 | 0.75 |
| value | p | old | y | 451 | 0.749 | 0.263 | 0.012 | 0.75 |
| value | p | yng | n | 454 | 0.702 | 0.350 | 0.016 | 0.75 |
| value | p | yng | o | 454 | 0.730 | 0.281 | 0.013 | 0.75 |
| value | p | yng | y | 454 | 0.777 | 0.250 | 0.012 | 0.75 |
Regarding negative behaviors:
Regarding positive behaviors:
So, it simply seems that people had better memory for the young than for the old target, and not sure whether there was any moderation by congruency between the behavior and the majority of behaviors performed by the target.
Let’s look at the same results with a different order, to see whether the positive target group affiliation conditions had any effect:
knitr::kable(mysumBy(value ~ who + positive + vlnc, dt = mmm))
| var | who | positive | vlnc | n | M | SD | SE | med |
|---|---|---|---|---|---|---|---|---|
| value | n | old | n | 451 | 0.729 | 0.356 | 0.016 | 1.00 |
| value | n | old | p | 451 | 0.684 | 0.358 | 0.017 | 0.75 |
| value | n | yng | n | 454 | 0.743 | 0.343 | 0.016 | 1.00 |
| value | n | yng | p | 454 | 0.702 | 0.350 | 0.016 | 0.75 |
| value | o | old | n | 451 | 0.774 | 0.289 | 0.013 | 1.00 |
| value | o | old | p | 451 | 0.713 | 0.280 | 0.013 | 0.75 |
| value | o | yng | n | 454 | 0.792 | 0.262 | 0.012 | 1.00 |
| value | o | yng | p | 454 | 0.730 | 0.281 | 0.013 | 0.75 |
| value | y | old | n | 451 | 0.835 | 0.246 | 0.011 | 1.00 |
| value | y | old | p | 451 | 0.749 | 0.263 | 0.012 | 0.75 |
| value | y | yng | n | 454 | 0.849 | 0.255 | 0.012 | 1.00 |
| value | y | yng | p | 454 | 0.777 | 0.250 | 0.012 | 0.75 |
Still hard to say.
We could get the conditions in, but probably best to see that in a plot. The top heading indicates the group of the postivie target. The next heading indicates the condition. The fill is the valence of the behavior. The x-axis is who was characterized by this behavior in the induction.
my.violin(DV = mmm$value, xFactor = mmm$who, fillFactor = mmm$vlnc, facet1 = mmm$positive, facet2 = mmm$condition)
The patterns might be different sometimes, but it’s still hard to tell.
So, perhaps we need an ANOVA to get a better idea about effects.
aov_ez(id = "session_id", dv = "value", data = mmm, within = c("who", "vlnc"), between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
The main effect of condition:
mysumBy(value ~ condition, dt = mmm)
Better memory when more individuating details were provided about the target in the induction. Providing the photo in the memory test doesn’t seem to improve memory.
Comparing all levels:
m1 <- emmeans(aaa, ~condition)
pairs(m1)
contrast estimate SE df t.ratio p.value
(name-only,name-only) - (name-only,name+photo) 0.0626 0.0203 895 3.091 0.0175
(name-only,name-only) - (name+age,name-only) -0.0695 0.0194 895 -3.579 0.0033
(name-only,name-only) - (name+age,name+photo) -0.0503 0.0198 895 -2.539 0.0831
(name-only,name-only) - (name+age+photo,name-only) -0.1057 0.0194 895 -5.436 <.0001
(name-only,name+photo) - (name+age,name-only) -0.1321 0.0198 895 -6.669 <.0001
(name-only,name+photo) - (name+age,name+photo) -0.1129 0.0202 895 -5.588 <.0001
(name-only,name+photo) - (name+age+photo,name-only) -0.1684 0.0198 895 -8.481 <.0001
(name+age,name-only) - (name+age,name+photo) 0.0191 0.0194 895 0.989 0.8605
(name+age,name-only) - (name+age+photo,name-only) -0.0363 0.0190 895 -1.911 0.3119
(name+age,name+photo) - (name+age+photo,name-only) -0.0554 0.0194 895 -2.856 0.0355
Results are averaged over the levels of: positive, vlnc, who
P value adjustment: tukey method for comparing a family of 5 estimates
Most of them significant.
Next, we found a main effect of the person who was actually characterized by the behavior:
mysumBy(value ~ who, dt = mmm)
Better for young targets.
Next, a main effect of valence of the behavior:
mysumBy(value ~ vlnc, dt = mmm)
More accurate for negative behaviors.
Next, the induction condition moderated the effect of who was actually characterized by the behavior:
knitr::kable(mysumBy(value ~ condition + who, dt = mmm))
| var | condition | who | n | M | SD | SE | med |
|---|---|---|---|---|---|---|---|
| value | name-only,name-only | n | 352 | 0.754 | 0.325 | 0.017 | 1.00 |
| value | name-only,name-only | o | 352 | 0.658 | 0.311 | 0.016 | 0.75 |
| value | name-only,name-only | y | 352 | 0.750 | 0.294 | 0.016 | 0.75 |
| value | name-only,name+photo | n | 326 | 0.613 | 0.398 | 0.022 | 0.75 |
| value | name-only,name+photo | o | 326 | 0.632 | 0.309 | 0.017 | 0.75 |
| value | name-only,name+photo | y | 326 | 0.729 | 0.277 | 0.015 | 0.75 |
| value | name+age,name-only | n | 390 | 0.749 | 0.333 | 0.017 | 1.00 |
| value | name+age,name-only | o | 390 | 0.799 | 0.255 | 0.013 | 1.00 |
| value | name+age,name-only | y | 390 | 0.822 | 0.247 | 0.012 | 1.00 |
| value | name+age,name+photo | n | 358 | 0.690 | 0.365 | 0.019 | 0.75 |
| value | name+age,name+photo | o | 358 | 0.798 | 0.231 | 0.012 | 0.75 |
| value | name+age,name+photo | y | 358 | 0.825 | 0.229 | 0.012 | 1.00 |
| value | name+age+photo,name-only | n | 384 | 0.753 | 0.322 | 0.016 | 1.00 |
| value | name+age+photo,name-only | o | 384 | 0.852 | 0.223 | 0.011 | 1.00 |
| value | name+age+photo,name-only | y | 384 | 0.873 | 0.207 | 0.010 | 1.00 |
No effects at all for the group affiliation of the positively portrayed target. People almost always had better memory for the young target’s behaviors, but memory for the old person’s behavior was not always better than for novel behaviors. Of particular note, in the [name-only, name-only] condition, the memory for the old target’s behavior was the worst, and the memory for the young target’s behaviors was not better than memory for the novel behaviors (of course, that could result from a bias to answer “none”, in that condition).
There were a couple of very small (but significant) effect. One was a three-way interaction between condition, who performed the behavior, and the behavior valence.
Let’s plot that interaction within each condition, to see where the difference is:
my.violin(DV = mmm$value, xFactor = mmm$who, fillFactor = mmm$vlnc, facet1 = mmm$condition)
I’m not sure. Perhaps if plot a bit differently:
my.violin(DV = mmm$value, xFactor = mmm$vlnc, fillFactor = mmm$who, facet1 = mmm$condition)
The patterns are different, but I don’t find a clear way to describe
anything interesting about it.
knitr::kable(mysumBy(value ~ condition + who + vlnc, dt = mmm))
| var | condition | who | vlnc | n | M | SD | SE | med |
|---|---|---|---|---|---|---|---|---|
| value | name-only,name-only | n | n | 176 | 0.780 | 0.313 | 0.023 | 1.00 |
| value | name-only,name-only | n | p | 176 | 0.727 | 0.336 | 0.025 | 0.75 |
| value | name-only,name-only | o | n | 176 | 0.723 | 0.315 | 0.023 | 0.75 |
| value | name-only,name-only | o | p | 176 | 0.594 | 0.294 | 0.022 | 0.75 |
| value | name-only,name-only | y | n | 176 | 0.776 | 0.297 | 0.022 | 1.00 |
| value | name-only,name-only | y | p | 176 | 0.724 | 0.290 | 0.022 | 0.75 |
| value | name-only,name+photo | n | n | 163 | 0.633 | 0.407 | 0.031 | 0.75 |
| value | name-only,name+photo | n | p | 163 | 0.592 | 0.389 | 0.030 | 0.75 |
| value | name-only,name+photo | o | n | 163 | 0.653 | 0.324 | 0.025 | 0.75 |
| value | name-only,name+photo | o | p | 163 | 0.610 | 0.293 | 0.023 | 0.75 |
| value | name-only,name+photo | y | n | 163 | 0.775 | 0.270 | 0.021 | 0.75 |
| value | name-only,name+photo | y | p | 163 | 0.683 | 0.278 | 0.021 | 0.75 |
| value | name+age,name-only | n | n | 195 | 0.781 | 0.322 | 0.023 | 1.00 |
| value | name+age,name-only | n | p | 195 | 0.718 | 0.342 | 0.024 | 0.75 |
| value | name+age,name-only | o | n | 195 | 0.833 | 0.243 | 0.017 | 1.00 |
| value | name+age,name-only | o | p | 195 | 0.764 | 0.263 | 0.019 | 0.75 |
| value | name+age,name-only | y | n | 195 | 0.842 | 0.257 | 0.018 | 1.00 |
| value | name+age,name-only | y | p | 195 | 0.801 | 0.235 | 0.017 | 0.75 |
| value | name+age,name+photo | n | n | 179 | 0.712 | 0.365 | 0.027 | 1.00 |
| value | name+age,name+photo | n | p | 179 | 0.668 | 0.365 | 0.027 | 0.75 |
| value | name+age,name+photo | o | n | 179 | 0.813 | 0.228 | 0.017 | 0.75 |
| value | name+age,name+photo | o | p | 179 | 0.784 | 0.233 | 0.017 | 0.75 |
| value | name+age,name+photo | y | n | 179 | 0.885 | 0.200 | 0.015 | 1.00 |
| value | name+age,name+photo | y | p | 179 | 0.764 | 0.240 | 0.018 | 0.75 |
| value | name+age+photo,name-only | n | n | 192 | 0.760 | 0.323 | 0.023 | 1.00 |
| value | name+age+photo,name-only | n | p | 192 | 0.746 | 0.322 | 0.023 | 0.75 |
| value | name+age+photo,name-only | o | n | 192 | 0.871 | 0.206 | 0.015 | 1.00 |
| value | name+age+photo,name-only | o | p | 192 | 0.832 | 0.237 | 0.017 | 1.00 |
| value | name+age+photo,name-only | y | n | 192 | 0.918 | 0.187 | 0.013 | 1.00 |
| value | name+age+photo,name-only | y | p | 192 | 0.828 | 0.217 | 0.015 | 1.00 |
Well, who knows. We could look at the whoXvalence interaction within each of the five conditions.
unique(mmm$condition)
[1] name+age,name+photo name+age,name-only name+age+photo,name-only name-only,name+photo name-only,name-only
Levels: name-only,name-only name-only,name+photo name+age,name-only name+age,name+photo name+age+photo,name-only
The confidence question was: How certain are you about this answer?‘, response options: ’guess’, ‘probably’, ‘certain’ (recoded as 1, 2, 3)
mcm <- mymelt(dt = allok, formula = conf.n.neg.1 + conf.n.neg.2 + conf.n.neg.3 + conf.n.pos.1 + conf.n.pos.2 + conf.n.pos.3 + conf.o.neg.1 + conf.o.neg.2 + conf.o.neg.3 + conf.o.pos.1 + conf.o.pos.2 + conf.o.pos.3 + conf.y.neg.1 + conf.y.neg.2 + conf.y.neg.3 + conf.y.pos.1 + conf.y.pos.2 + conf.y.pos.3 ~ session_id + condition + positive)
mcm$who <- substr(mcm$variable, 6, 6)
mcm$vlnc <- substr(mcm$variable, 8, 8)
mcm$conf <- substr(mcm$variable, 12, 12)
mcm <- mcm[complete.cases(mcm), ]
Memory accuracy by confidence:
mysumBy(value ~ conf, dt = mcm)
As can be expected, more accuracy when people are more confident about their response.
It is possible that a bias to respond “young” caused the higher accuracy rate regarding the young target. To examine that possibility, we will start by examining the number of responses.
mmm <- mymelt(formula = m.n.byresp.non.neg + m.n.byresp.non.pos + m.n.byresp.yng.neg + m.n.byresp.yng.pos + m.n.byresp.old.neg + m.n.byresp.old.pos ~ session_id + condition + positive, dt = allok)
mmm$resp <- substr(mmm$variable, 12, 14)
mmm$vlnc <- substr(mmm$variable, 16, 16)
mmm$n.resps <- ifelse(is.na(mmm$value), 0, mmm$value)
mysumBy(n.resps ~ resp + vlnc, dt = mmm)
We see a clear bias toward responding “yng” and against responding “old”.
Plot:
my.violin(DV = mmm$n.resps, xFactor = mmm$vlnc, fillFactor = mmm$resp)
The young-response-bias effect seems larger for negative than for
positive behaviors.
Let’s see this also with positive-target group:
my.violin(DV = mmm$n.resps, xFactor = mmm$vlnc, fillFactor = mmm$resp, facet1 = mmm$positive)
When the positively-portrayed target individuals was the old man, the
young-response-bias was stronger for negative than for positive
behaviors. When the positively-portrayed target individuals was the
young man, the young-response-bias was stronger for positive than for
negative behaviors. That means that people used the consistency between
the target’s valence and the behavior valence as a memory-cue. That
makese sense, of course.
Let’s ANOVA this with the other manipulated factors:
aov_ez(id = "session_id", dv = "n.resps", data = mmm, within = c("resp", "vlnc"), between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
A strong effect of response:
mysumBy(n.resps ~ resp, dt = mmm)
A small effect for the three-way interaction between response, positively-protrayed target’s group, and behavior valence. It is the valence compatibilty effect we noticed earlier:
knitr::kable(mysumBy(n.resps ~ resp + positive + vlnc, dt = mmm))
| var | resp | positive | vlnc | n | M | SD | SE | med |
|---|---|---|---|---|---|---|---|---|
| n.resps | non | old | n | 466 | 3.124 | 1.676 | 0.078 | 4 |
| n.resps | non | old | p | 466 | 3.251 | 1.880 | 0.087 | 4 |
| n.resps | non | yng | n | 459 | 3.246 | 1.526 | 0.071 | 4 |
| n.resps | non | yng | p | 459 | 3.279 | 1.744 | 0.081 | 4 |
| n.resps | old | old | n | 466 | 3.764 | 1.491 | 0.069 | 4 |
| n.resps | old | old | p | 466 | 4.159 | 1.866 | 0.086 | 4 |
| n.resps | old | yng | n | 459 | 4.155 | 1.543 | 0.072 | 4 |
| n.resps | old | yng | p | 459 | 3.908 | 1.551 | 0.072 | 4 |
| n.resps | yng | old | n | 466 | 4.627 | 1.860 | 0.086 | 4 |
| n.resps | yng | old | p | 466 | 4.116 | 1.711 | 0.079 | 4 |
| n.resps | yng | yng | n | 459 | 4.403 | 1.549 | 0.072 | 4 |
| n.resps | yng | yng | p | 459 | 4.632 | 1.811 | 0.085 | 4 |
A small interaction between the condition and the response:
knitr::kable(mysumBy(n.resps ~ condition + resp, dt = mmm))
| var | condition | resp | n | M | SD | SE | med |
|---|---|---|---|---|---|---|---|
| n.resps | name-only,name-only | non | 360 | 3.472 | 1.620 | 0.085 | 4 |
| n.resps | name-only,name-only | old | 360 | 3.714 | 1.810 | 0.095 | 4 |
| n.resps | name-only,name-only | yng | 360 | 4.492 | 1.929 | 0.102 | 4 |
| n.resps | name-only,name+photo | non | 338 | 2.754 | 1.923 | 0.105 | 3 |
| n.resps | name-only,name+photo | old | 338 | 3.920 | 1.784 | 0.097 | 4 |
| n.resps | name-only,name+photo | yng | 338 | 4.678 | 2.035 | 0.111 | 4 |
| n.resps | name+age,name-only | non | 396 | 3.384 | 1.690 | 0.085 | 4 |
| n.resps | name+age,name-only | old | 396 | 4.053 | 1.475 | 0.074 | 4 |
| n.resps | name+age,name-only | yng | 396 | 4.351 | 1.588 | 0.080 | 4 |
| n.resps | name+age,name+photo | non | 364 | 3.096 | 1.658 | 0.087 | 4 |
| n.resps | name+age,name+photo | old | 364 | 4.212 | 1.682 | 0.088 | 4 |
| n.resps | name+age,name+photo | yng | 364 | 4.462 | 1.724 | 0.090 | 4 |
| n.resps | name+age+photo,name-only | non | 392 | 3.362 | 1.585 | 0.080 | 4 |
| n.resps | name+age+photo,name-only | old | 392 | 4.064 | 1.349 | 0.068 | 4 |
| n.resps | name+age+photo,name-only | yng | 392 | 4.276 | 1.443 | 0.073 | 4 |
When photos appears in the memory test, people were less likely to choose “none” than in the other conditions.
This 2-way interaction was further moderated by the behavior valence:
knitr::kable(mysumBy(n.resps ~ vlnc + resp + condition, dt = mmm))
| var | vlnc | resp | condition | n | M | SD | SE | med |
|---|---|---|---|---|---|---|---|---|
| n.resps | n | non | name-only,name-only | 180 | 3.394 | 1.470 | 0.110 | 4 |
| n.resps | n | non | name-only,name+photo | 169 | 2.716 | 1.787 | 0.137 | 3 |
| n.resps | n | non | name+age,name-only | 198 | 3.449 | 1.639 | 0.116 | 4 |
| n.resps | n | non | name+age,name+photo | 182 | 3.055 | 1.551 | 0.115 | 4 |
| n.resps | n | non | name+age+photo,name-only | 196 | 3.250 | 1.479 | 0.106 | 4 |
| n.resps | n | old | name-only,name-only | 180 | 3.928 | 1.834 | 0.137 | 4 |
| n.resps | n | old | name-only,name+photo | 169 | 3.722 | 1.680 | 0.129 | 4 |
| n.resps | n | old | name+age,name-only | 198 | 4.056 | 1.307 | 0.093 | 4 |
| n.resps | n | old | name+age,name+photo | 182 | 4.011 | 1.534 | 0.114 | 4 |
| n.resps | n | old | name+age+photo,name-only | 196 | 4.041 | 1.256 | 0.090 | 4 |
| n.resps | n | yng | name-only,name-only | 180 | 4.350 | 1.754 | 0.131 | 4 |
| n.resps | n | yng | name-only,name+photo | 169 | 4.917 | 2.057 | 0.158 | 5 |
| n.resps | n | yng | name+age,name-only | 198 | 4.273 | 1.569 | 0.112 | 4 |
| n.resps | n | yng | name+age,name+photo | 182 | 4.687 | 1.739 | 0.129 | 4 |
| n.resps | n | yng | name+age+photo,name-only | 196 | 4.408 | 1.384 | 0.099 | 4 |
| n.resps | p | non | name-only,name-only | 180 | 3.550 | 1.757 | 0.131 | 4 |
| n.resps | p | non | name-only,name+photo | 169 | 2.793 | 2.055 | 0.158 | 3 |
| n.resps | p | non | name+age,name-only | 198 | 3.318 | 1.741 | 0.124 | 4 |
| n.resps | p | non | name+age,name+photo | 182 | 3.137 | 1.761 | 0.131 | 4 |
| n.resps | p | non | name+age+photo,name-only | 196 | 3.474 | 1.681 | 0.120 | 4 |
| n.resps | p | old | name-only,name-only | 180 | 3.500 | 1.764 | 0.131 | 4 |
| n.resps | p | old | name-only,name+photo | 169 | 4.118 | 1.867 | 0.144 | 4 |
| n.resps | p | old | name+age,name-only | 198 | 4.051 | 1.630 | 0.116 | 4 |
| n.resps | p | old | name+age,name+photo | 182 | 4.412 | 1.799 | 0.133 | 4 |
| n.resps | p | old | name+age+photo,name-only | 196 | 4.087 | 1.439 | 0.103 | 4 |
| n.resps | p | yng | name-only,name-only | 180 | 4.633 | 2.085 | 0.155 | 4 |
| n.resps | p | yng | name-only,name+photo | 169 | 4.438 | 1.990 | 0.153 | 4 |
| n.resps | p | yng | name+age,name-only | 198 | 4.429 | 1.607 | 0.114 | 4 |
| n.resps | p | yng | name+age,name+photo | 182 | 4.236 | 1.683 | 0.125 | 4 |
| n.resps | p | yng | name+age+photo,name-only | 196 | 4.143 | 1.492 | 0.107 | 4 |
For negative behaviors, the “none” response was less likely and the “young” response was more likely when photos appeared in the memory test. For positive behaviors, the “none” response was less likely and the “old” response was more likely when photos appeared in the memory test. I am not sure what that might mean.
Let’s see separate ANOVAs. For negative behaviors only:
aov_ez(id = "session_id", dv = "n.resps", data = mmm[which(mmm$vlnc == "n"), ], within = c("resp"), between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
for positive behavior only:
aov_ez(id = "session_id", dv = "n.resps", data = mmm[which(mmm$vlnc == "p"), ], within = c("resp"), between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
The conditionXresp interaction effects are of the same size, but we saw they were different when we unpacked the 3-way interaction above.
mysumBy(n.resps ~ vlnc + resp, dt = mmm)
So, all and all, it could be that people’s tendency to respond “young” caused the higher accuracy rates for behavior of the young target than behaviors of the old target (and for novel behaviors). But, there’s another way to test that, nex…
Let’s see whether there are differences in memory accuracy between different responses (rather than betwen behaviors of the young target vs. old target vs. novel behaviors)
mmm <- mymelt(formula = m.acc.byresp.non.neg + m.acc.byresp.non.pos + m.acc.byresp.yng.neg + m.acc.byresp.yng.pos + m.acc.byresp.old.neg + m.acc.byresp.old.pos ~ session_id + condition + positive, dt = allok)
mmm$resp <- substr(mmm$variable, 14, 16)
mmm$vlnc <- substr(mmm$variable, 18, 18)
mysumBy(value ~ resp + vlnc, dt = mmm)
Indeed, we no longer see better accuracy for the young response. Rather, the best accuracy is for novel behaviors. This is the mirror image of the response frequency. It makes some sense because
DOES IT MEAN WE NEED TO CONDUCT AN SDT ANALYSIS OF SOME SORTS?
alphas <- list()
alphas$self.report.negative.attributes.old <- round(psych::alpha(allok[, c("eval.oneg1", "eval.oneg2", "eval.oneg3")])$total$std.alpha, 2)
alphas$self.report.negative.attributes.young <- round(psych::alpha(allok[, c("eval.yneg1", "eval.yneg2", "eval.yneg3")])$total$std.alpha, 2)
alphas$self.report.positive.attributes.old <- round(psych::alpha(allok[, c("eval.opos1", "eval.opos2", "eval.opos3")])$total$std.alpha, 2)
alphas$self.report.positive.attributes.young <- round(psych::alpha(allok[, c("eval.ypos1", "eval.ypos2", "eval.ypos3")])$total$std.alpha, 2)
alphas$IAT.alpha <- round(psych::alpha(allok[, c("IAT1", "IAT2")])$total$std.alpha, 2)
alphas
$self.report.negative.attributes.old
[1] 0.82
$self.report.negative.attributes.young
[1] 0.79
$self.report.positive.attributes.old
[1] 0.87
$self.report.positive.attributes.young
[1] 0.88
$IAT.alpha
[1] 0.6
Quite typical for these evaluation measures.
In the quick results section, we used the variable eval.diff which was computed as the difference of differences
allok$posTarget <- ifelse(allok$positive == "yng", 1, ifelse(allok$positive == "old", 0, NA))
So, there are a few self-reported evaluation scores here, and we can examine the correlation between them [posTarget is 1 if young, 0 if old; IAT.prf is the IAT score recoded to reflect a preference for the young individual over the old individual]:
my.htmlTable(cornp(allok[, c("ypos", "yneg", "opos", "oneg", "IAT.prf", "posTarget")]))
| varName____ | ypos____ | yneg____ | opos____ | oneg____ | IAT.prf____ | |
|---|---|---|---|---|---|---|
| 1 | yneg |
-0.336 < .001 925 |
||||
| 2 | opos |
0.047 0.153 925 |
0.344 < .001 925 |
|||
| 3 | oneg |
0.335 < .001 925 |
0.162 < .001 925 |
-0.381 < .001 925 |
||
| 4 | IAT.prf |
0.151 < .001 925 |
-0.107 0.001 925 |
-0.197 < .001 925 |
0.144 < .001 925 |
|
| 5 | posTarget |
0.352 < .001 925 |
-0.23 < .001 925 |
-0.333 < .001 925 |
0.314 < .001 925 |
0.197 < .001 925 |
ok. These results might mean that the behaviors (i.e., the attitude induction) had a weaker influence on the IAT score and the negative-attributes ratings of the young target than on the other evaluation measures (the differences are significant).
We’ll analyze the self-report evaluation with an attribute-valence factor (rather than the difference score). Because these are many factors, it will be easier to start with the ANOVA, and use that for guiding our attention to effects.
mmm <- mymelt(formula = ypos + yneg + opos + oneg ~ session_id + condition + positive, dt = allok)
mmm$group <- substr(mmm$variable, 1, 1)
mmm$att.vlnc <- substr(mmm$variable, 2, 5)
rrr <- aov_ez(id = "session_id", dv = "value", data = mmm, within = c("group", "att.vlnc"), between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
print(rrr)
Anova Table (Type 3 tests)
Response: value
Effect df MSE F pes p.value
1 condition 4, 915 498.73 0.10 <.001 .983
2 positive 1, 915 498.73 2.60 .003 .107
3 condition:positive 4, 915 498.73 0.12 <.001 .975
4 group 1, 915 87.85 1.53 .002 .216
5 condition:group 4, 915 87.85 0.42 .002 .793
6 positive:group 1, 915 87.85 19.52 *** .021 <.001
7 condition:positive:group 4, 915 87.85 1.31 .006 .263
8 att.vlnc 1, 915 509.41 22.28 *** .024 <.001
9 condition:att.vlnc 4, 915 509.41 0.93 .004 .445
10 positive:att.vlnc 1, 915 509.41 1.42 .002 .234
11 condition:positive:att.vlnc 4, 915 509.41 2.61 * .011 .034
12 group:att.vlnc 1, 915 552.42 6.17 * .007 .013
13 condition:group:att.vlnc 4, 915 552.42 2.15 + .009 .073
14 positive:group:att.vlnc 1, 915 552.42 283.82 *** .237 <.001
15 condition:positive:group:att.vlnc 4, 915 552.42 0.98 .004 .417
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '+' 0.1 ' ' 1
positive = the group of the positively-portrayed target individual; group = the group of the evaluated target individual (i.e., whether participants the evaluation pertained to the young person or the old person); att.vlnc = whether the evaluation was for positive attributes or negative attributes.;
The largest effect is a 3-way interaction between the group of the positively-portrayed target, the evaluated group and the valence of the evaluation attributes. It probably simply reflects the effect of the induction: The relevant means:
mysumBy(value ~ positive + group + att.vlnc, dt = mmm)
Yes, that is the induction effect: when the positively-portrayed target was old (positive=old), participants reported that positive attributes characterize the old target more than negative attributes and the opposite for the young target. The pattern was the complete opposite when the positively-portrayed target was young (positive=yng).
The next effect was a moderate main effect of attribute valence:
mysumBy(value ~ att.vlnc, dt = mmm)
Participants thought the positive attributes chracterized the target individuals more than negative attributes. A positivity-bias.
Next, an interaction between the group of the evaluated individual (“group”) and the group of positively-portrayed target (“positive”)
mysumBy(value ~ positive + group, dt = mmm)
Well, because the attribute valence is not involved in this effect, this is just an odd effect reflecting higher rating (regradless of attribute valence) in the positive=young than in the positive=old condition.
Next, a small 3-way interaction of condition by positive by att.vlnc:
knitr::kable(mysumBy(value ~ condition + positive + att.vlnc, dt = mmm))
| var | condition | positive | att.vlnc | n | M | SD | SE | med |
|---|---|---|---|---|---|---|---|---|
| value | name-only,name-only | old | neg | 178 | 37.934 | 22.927 | 1.718 | 34.167 |
| value | name-only,name-only | old | pos | 178 | 43.408 | 22.642 | 1.697 | 45.000 |
| value | name-only,name-only | yng | neg | 182 | 42.332 | 22.505 | 1.668 | 43.333 |
| value | name-only,name-only | yng | pos | 182 | 42.973 | 21.507 | 1.594 | 45.833 |
| value | name-only,name+photo | old | neg | 174 | 37.559 | 22.583 | 1.712 | 37.333 |
| value | name-only,name+photo | old | pos | 174 | 44.661 | 22.997 | 1.743 | 44.333 |
| value | name-only,name+photo | yng | neg | 164 | 42.240 | 20.371 | 1.591 | 42.667 |
| value | name-only,name+photo | yng | pos | 164 | 41.026 | 19.506 | 1.523 | 41.000 |
| value | name+age,name-only | old | neg | 216 | 38.110 | 20.683 | 1.407 | 38.167 |
| value | name+age,name-only | old | pos | 216 | 43.174 | 19.819 | 1.349 | 45.333 |
| value | name+age,name-only | yng | neg | 180 | 40.480 | 21.760 | 1.622 | 38.667 |
| value | name+age,name-only | yng | pos | 180 | 43.567 | 21.225 | 1.582 | 44.667 |
| value | name+age,name+photo | old | neg | 166 | 39.201 | 21.071 | 1.635 | 39.333 |
| value | name+age,name+photo | old | pos | 166 | 41.946 | 21.045 | 1.633 | 45.000 |
| value | name+age,name+photo | yng | neg | 198 | 36.768 | 21.424 | 1.523 | 36.667 |
| value | name+age,name+photo | yng | pos | 198 | 45.813 | 20.633 | 1.466 | 48.833 |
| value | name+age+photo,name-only | old | neg | 198 | 39.778 | 21.215 | 1.508 | 41.333 |
| value | name+age+photo,name-only | old | pos | 198 | 41.402 | 21.501 | 1.528 | 43.667 |
| value | name+age+photo,name-only | yng | neg | 194 | 41.136 | 21.291 | 1.529 | 41.500 |
| value | name+age+photo,name-only | yng | pos | 194 | 42.711 | 20.591 | 1.478 | 45.333 |
This is probably not very meaningful because the means are beyond the evaluated individual (i.e., whether he’s old or young).
Finally, a tiny group by att.vlnc effect. This could reflect a pro-young bias, beyond the behavioral information provided about the targets (the “positive” factor)
mysumBy(value ~ group + att.vlnc, dt = mmm)
Yes, a tiny bias in favor of the young target. With eval.gb (i.e., the difference score), the effect had exactly the same (tiny) size. ok.
First, a reminder about the language variable: Language. We asked: Which of the following is the most accurate? answers: [‘English is my native language’, ‘English is not my native langauge, but it is my primary language’, ‘English is not my native language, and not my primary language, but I am fluent in English’, ‘I understand some English’ ] coded as 1, 2, 3, and 4 (-999 selected a “decline to answer” button at the bottom of the page)
my.freq(allok$lang)
Let’s see whether the results are mighty different if we include only participants who reported that English was their native language.
We’ll jump to the ANOVA, to see whether anything of interest is detected here
aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$lang == 1), ], between = c("iat.order1", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))
The effect of “positive” (the group of the positively-portrayed target) was half of the small effect found with the whole sample. So, let’s see whether the effect becomes stronger the less people understand English:
mysumBy(IAT.gb ~ lang + positive, dt = allok)
Well, it seems that the effect was always in the expected direction, but clearly more tiny in the largest group, those who reported that English is their native language. Oddly, they also showed a relatively small induction effect (preference for the positively portrayed target individual). So, it is not entirely clear whether this moderation by language may reflect a stronger reliance on people’s groups than on their behaviors the less they know English.
The same for the self-report evaluation measure:
mysumBy(eval.gb ~ lang + positive, dt = allok)
The IAT pattern does not replicate at all. The largest effects are found in the two conditions that showed the smallest effect with the IAT.
ANOVA with language as a factor. For the IAT:
aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$lang %in% c(1, 2, 3, 4)), ], between = c("condition", "positive", "lang"), return = "nice", anova_table = list(es = "pes"))
Nothing. We lost the small bias effect (of the factor “positive”).
For self-report:
aov_ez(id = "session_id", dv = "eval.gb", data = allok[which(allok$lang %in% c(1, 2, 3, 4)), ], between = c("condition", "positive", "lang"), return = "nice", anova_table = list(es = "pes"))
Again, no effect at all, after losing the tiny effect of “positive”.
Let’s focus on the effect that showed the stronger pro-young bias with the IAT (name+age,name+photo):
mysumBy(IAT.gb ~ lang + positive, dt = allok[which(allok$condition == "name+age,name+photo"), ])
The Ns are very small, but it seems that the effect was found in all the language groups, other than lang=2 (probably random, due to small number of participants).
First, here are the percentiles of the participants’ report age:
plot.percentile(allok$age)
Probably people under 50 (80%) would feel closer to the young person
than to the old people.
We will start with the relation between age and preference for the young target over the old target, and a bunch of memory scores [macc is the accuracy rate by the person who was actually attributed with the behavior and the valence of the behavior, m.acc.byresp is the memory accuracy rate by the participant’s responseand the valence of the behavior, m.n.byresp is the number of responses of each type of response and the valence of the behavior]:
my.htmlTable(cornp(allok[, c("age", "eval.diff", "IAT.prf", "macc.o.neg", "macc.o.pos", "macc.y.neg", "macc.y.pos", "m.n.byresp.old.neg", "m.n.byresp.old.pos", "m.n.byresp.yng.neg", "m.n.byresp.yng.pos", "m.acc.byresp.old.neg", "m.acc.byresp.old.pos", "m.acc.byresp.yng.neg", "m.acc.byresp.yng.pos")]))
| varName____ | age____ | eval.diff____ | IAT.prf____ | macc.o.neg____ | macc.o.pos____ | macc.y.neg____ | macc.y.pos____ | m.n.byresp.old.neg____ | m.n.byresp.old.pos____ | m.n.byresp.yng.neg____ | m.n.byresp.yng.pos____ | m.acc.byresp.old.neg____ | m.acc.byresp.old.pos____ | m.acc.byresp.yng.neg____ | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | eval.diff |
0.021 0.523 925 |
|||||||||||||
| 2 | IAT.prf |
0.007 0.831 925 |
0.237 < .001 925 |
||||||||||||
| 3 | macc.o.neg |
-0.015 0.65 905 |
0.04 0.233 905 |
-0.051 0.129 905 |
|||||||||||
| 4 | macc.o.pos |
-0.021 0.536 905 |
-0.067 0.043 905 |
-0.056 0.093 905 |
0.366 < .001 905 |
||||||||||
| 5 | macc.y.neg |
-0.072 0.029 905 |
-0.059 0.076 905 |
-0.024 0.465 905 |
0.437 < .001 905 |
0.515 < .001 905 |
|||||||||
| 6 | macc.y.pos |
0.018 0.584 905 |
0.049 0.14 905 |
-0.066 0.049 905 |
0.492 < .001 905 |
0.327 < .001 905 |
0.332 < .001 905 |
||||||||
| 7 | m.n.byresp.old.neg |
0.06 0.072 896 |
0.219 < .001 896 |
-0.007 0.835 896 |
0.443 < .001 896 |
-0.11 < .001 896 |
-0.341 < .001 896 |
0.202 < .001 896 |
|||||||
| 8 | m.n.byresp.old.pos |
0.006 0.863 892 |
-0.167 < .001 892 |
-0.015 0.658 892 |
-0.176 < .001 892 |
0.465 < .001 892 |
0.082 0.014 892 |
-0.357 < .001 892 |
-0.129 < .001 885 |
||||||
| 9 | m.n.byresp.yng.neg |
-0.016 0.63 898 |
-0.166 < .001 898 |
-0.005 0.891 898 |
-0.463 < .001 898 |
0.081 0.015 898 |
0.279 < .001 898 |
-0.197 < .001 898 |
-0.462 < .001 890 |
0.479 < .001 888 |
|||||
| 10 | m.n.byresp.yng.pos |
0.079 0.018 898 |
0.215 < .001 898 |
0.016 0.635 898 |
0.033 0.317 898 |
-0.414 < .001 898 |
-0.222 < .001 898 |
0.408 < .001 898 |
0.461 < .001 893 |
-0.392 < .001 887 |
-0.021 0.531 893 |
||||
| 11 | m.acc.byresp.old.neg |
-0.061 0.069 896 |
-0.122 < .001 896 |
-0.041 0.223 896 |
0.607 < .001 896 |
0.425 < .001 896 |
0.7 < .001 896 |
0.304 < .001 896 |
-0.368 < .001 896 |
-0.073 0.03 885 |
-0.092 0.006 890 |
-0.339 < .001 893 |
|||
| 12 | m.acc.byresp.old.pos |
-0.022 0.514 892 |
0.075 0.025 892 |
-0.044 0.192 892 |
0.501 < .001 892 |
0.539 < .001 892 |
0.469 < .001 892 |
0.657 < .001 892 |
-0.015 0.663 885 |
-0.412 < .001 892 |
-0.304 < .001 888 |
-0.034 0.309 887 |
0.509 < .001 885 |
||
| 13 | m.acc.byresp.yng.neg |
-0.045 0.178 898 |
0.074 0.027 898 |
-0.002 0.95 898 |
0.729 < .001 898 |
0.365 < .001 898 |
0.623 < .001 898 |
0.42 < .001 898 |
0.064 0.057 890 |
-0.278 < .001 888 |
-0.509 < .001 898 |
-0.153 < .001 893 |
0.649 < .001 890 |
0.618 < .001 888 |
|
| 14 | m.acc.byresp.yng.pos |
-0.05 0.132 898 |
-0.135 < .001 898 |
-0.076 0.023 898 |
0.415 < .001 898 |
0.643 < .001 898 |
0.477 < .001 898 |
0.534 < .001 898 |
-0.188 < .001 893 |
0.015 0.651 887 |
-0.162 < .001 893 |
-0.482 < .001 898 |
0.578 < .001 893 |
0.596 < .001 887 |
0.518 < .001 893 |
Age was unrelated to any score. IAT and self-report were weakly related (as often found with age attitudes) The relations of the self-reported evaluation with a couple of the memory scores is probably random, considering that it was unrelated to other memory scores.
Let’s see whether restricting the sample to age<50 would change the results. The IAT anova:
aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$age < 50), ], between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
The effect is similar, but it is so tiny that it is non-significant.
Only age>49:
aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$age > 49), ], between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
The effect is large, but still non-significant (smaller sample). So, if anything, there is a larger effect with older participants, but it may also be due to random assignment to different conditions.
Age by condition:
mysumBy(age ~ condition, dt = allok, probs = c(0.25, 0.75))
It’s hard to tell.
Age as a covariate (IAT.gb ~ condition * positive + age):
ccc <- allok[, c("session_id", "IAT.gb", "condition", "positive", "age")]
ccc <- ccc[complete.cases(ccc), ]
library(car)
# Fit the linear model
lm_model <- lm(IAT.gb ~ condition * positive + age, data = ccc)
# Perform ANOVA
anova_result <- Anova(lm_model, type = 3)
# Print the ANOVA table
print(anova_result)
Anova Table (Type III tests)
Response: IAT.gb
Sum Sq Df F value Pr(>F)
(Intercept) 0.646 1 4.3153 0.03805 *
condition 2.459 4 4.1052 0.00265 **
positive 0.026 1 0.1768 0.67424
age 0.016 1 0.1072 0.74338
condition:positive 0.983 4 1.6408 0.16186
Residuals 136.868 914
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
But, I am not sure that we are expect age to really predict the preference between the positive and negative targets (by their behaviors), and therefore, just adding age as a covariate may not make much sense.
First, recoded gender to male=0, female-1, and correlated with the evaluation measures and the memory scores
allok$gender.i <- ifelse(allok$gender == "f", 1, ifelse(allok$gender == "m", 0, NA))
my.htmlTable(cornp(allok[, c("gender.i", "eval.diff", "IAT.prf", "macc.o.neg", "macc.o.pos", "macc.y.neg", "macc.y.pos", "m.n.byresp.old.neg", "m.n.byresp.old.pos", "m.n.byresp.yng.neg", "m.n.byresp.yng.pos", "m.acc.byresp.old.neg", "m.acc.byresp.old.pos", "m.acc.byresp.yng.neg", "m.acc.byresp.yng.pos")]))
| varName____ | gender.i____ | eval.diff____ | IAT.prf____ | macc.o.neg____ | macc.o.pos____ | macc.y.neg____ | macc.y.pos____ | m.n.byresp.old.neg____ | m.n.byresp.old.pos____ | m.n.byresp.yng.neg____ | m.n.byresp.yng.pos____ | m.acc.byresp.old.neg____ | m.acc.byresp.old.pos____ | m.acc.byresp.yng.neg____ | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | eval.diff |
-0.066 0.049 887 |
|||||||||||||
| 2 | IAT.prf |
-0.048 0.149 887 |
0.237 < .001 925 |
||||||||||||
| 3 | macc.o.neg |
0.015 0.651 868 |
0.04 0.233 905 |
-0.051 0.129 905 |
|||||||||||
| 4 | macc.o.pos |
0.091 0.008 868 |
-0.067 0.043 905 |
-0.056 0.093 905 |
0.366 < .001 905 |
||||||||||
| 5 | macc.y.neg |
0.069 0.043 868 |
-0.059 0.076 905 |
-0.024 0.465 905 |
0.437 < .001 905 |
0.515 < .001 905 |
|||||||||
| 6 | macc.y.pos |
0.011 0.751 868 |
0.049 0.14 905 |
-0.066 0.049 905 |
0.492 < .001 905 |
0.327 < .001 905 |
0.332 < .001 905 |
||||||||
| 7 | m.n.byresp.old.neg |
-0.039 0.252 859 |
0.219 < .001 896 |
-0.007 0.835 896 |
0.443 < .001 896 |
-0.11 < .001 896 |
-0.341 < .001 896 |
0.202 < .001 896 |
|||||||
| 8 | m.n.byresp.old.pos |
0.099 0.004 856 |
-0.167 < .001 892 |
-0.015 0.658 892 |
-0.176 < .001 892 |
0.465 < .001 892 |
0.082 0.014 892 |
-0.357 < .001 892 |
-0.129 < .001 885 |
||||||
| 9 | m.n.byresp.yng.neg |
0.066 0.053 861 |
-0.166 < .001 898 |
-0.005 0.891 898 |
-0.463 < .001 898 |
0.081 0.015 898 |
0.279 < .001 898 |
-0.197 < .001 898 |
-0.462 < .001 890 |
0.479 < .001 888 |
|||||
| 10 | m.n.byresp.yng.pos |
-0.035 0.299 861 |
0.215 < .001 898 |
0.016 0.635 898 |
0.033 0.317 898 |
-0.414 < .001 898 |
-0.222 < .001 898 |
0.408 < .001 898 |
0.461 < .001 893 |
-0.392 < .001 887 |
-0.021 0.531 893 |
||||
| 11 | m.acc.byresp.old.neg |
0.043 0.211 859 |
-0.122 < .001 896 |
-0.041 0.223 896 |
0.607 < .001 896 |
0.425 < .001 896 |
0.7 < .001 896 |
0.304 < .001 896 |
-0.368 < .001 896 |
-0.073 0.03 885 |
-0.092 0.006 890 |
-0.339 < .001 893 |
|||
| 12 | m.acc.byresp.old.pos |
0.009 0.791 856 |
0.075 0.025 892 |
-0.044 0.192 892 |
0.501 < .001 892 |
0.539 < .001 892 |
0.469 < .001 892 |
0.657 < .001 892 |
-0.015 0.663 885 |
-0.412 < .001 892 |
-0.304 < .001 888 |
-0.034 0.309 887 |
0.509 < .001 885 |
||
| 13 | m.acc.byresp.yng.neg |
-0.023 0.504 861 |
0.074 0.027 898 |
-0.002 0.95 898 |
0.729 < .001 898 |
0.365 < .001 898 |
0.623 < .001 898 |
0.42 < .001 898 |
0.064 0.057 890 |
-0.278 < .001 888 |
-0.509 < .001 898 |
-0.153 < .001 893 |
0.649 < .001 890 |
0.618 < .001 888 |
|
| 14 | m.acc.byresp.yng.pos |
0.02 0.554 861 |
-0.135 < .001 898 |
-0.076 0.023 898 |
0.415 < .001 898 |
0.643 < .001 898 |
0.477 < .001 898 |
0.534 < .001 898 |
-0.188 < .001 893 |
0.015 0.651 887 |
-0.162 < .001 893 |
-0.482 < .001 898 |
0.578 < .001 893 |
0.596 < .001 887 |
0.518 < .001 893 |
Gender is unrelated to anything.
The main IAT anova, among women only:
aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$gender == "f"), ], between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
A small effect of the condition. It is probably not very interesting, but let’s see it:
mysumBy(IAT.gb ~ condition, dt = allok[which(allok$gender == "f"), ])
Very small effects of the induction, and the differences are hard to interpret.
The same ANOVA with males:
aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$gender == "m"), ], between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
The men showed the bias, with a stronger effect and a smaller sample. Perhaps because they matched on gender (i.e., the targets were males). We haven’t tested that in the original study.
Means of IAT preference for the positively-portrayed individual by condition, males only:
mysumBy(IAT.gb ~ condition + positive, dt = allok[which(allok$gender == "m"), ])
The bias shows in the baseline condition, and in the two conditions that showed photos in the measurement.
Adding gender to the main ANOVA, as a factor.
aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$gender %in% c("m", "f")), ], between = c("gender", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))
Gender does not moderate anything, but it does make the bias effect significant.
The main anova, among women only:
aov_ez(id = "session_id", dv = "eval.gb", data = allok[which(allok$gender == "f"), ], between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
Nada.
Men only:
aov_ez(id = "session_id", dv = "eval.gb", data = allok[which(allok$gender == "m"), ], between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
The bias effect is as strong as with the IAT.
Let’s see it by condition:
mysumBy(eval.gb ~ condition + positive, dt = allok[which(allok$gender == "m"), ])
The bias shows in all the conditions, other than [name-only,name+photo], and it is the strongest in [name+age+photo,name-only].
Adding gender to the main ANOVA, as a factor.
aov_ez(id = "session_id", dv = "eval.gb", data = allok[which(allok$gender %in% c("m", "f")), ], between = c("gender", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))
Gender does not moderate anything, and the bias effect remains significant, but tiny.
We saved the conditions of the study within different tasks. Other times, it is possible to infer the condition from the data. For example, if we saved a photo data in the IAT, then we can infer that the paricipant observed a photo. There could be some inconsistencies due to technical problems. I think that the main reason is refreshing the page. It is supposed to cause an error an kill the study, but perhaps clicking Back in the browser, sometimes restarts the study session, possibly with different conditions. In this section, we will examine whether the conditions saved in the sample are consistent.
In the induction task, we saved the condition by which we determined what to show to participants. Let’s make sure it was the same condition as the one recorded at the begining of the task.
table(indPart1Cond = allok$indPart1Cond, part1Cond = allok$part1Cond, exclude = NULL)
part1Cond
indPart1Cond name-only name+age name+age+photo
name-only 349 0 0
name+age 0 380 0
name+age+photo 0 0 196
Perfect.
From the induction, we can infer whether the old or the yng target individual was the positive or the negative target, based on the number of positive (and negative) behaviors presented in the induction task
table(ind.oGood = allok$ind.oGood, oldVlnc = allok$oldVlnc, exclude = NULL)
oldVlnc
ind.oGood b g
FALSE 459 0
TRUE 0 466
Good. and for the young:
table(ind.yGood = allok$ind.yGood, oldVlnc = allok$oldVlnc, exclude = NULL)
oldVlnc
ind.yGood b g
FALSE 0 466
TRUE 459 0
Good.
I also counted the number of positive and negative behaviors presented in the induction, based on the data recorded in that task. Old target:
table(old.bvs = allok$o.bvs, young.bvs = allok$y.bvs, exclude = NULL)
young.bvs
old.bvs 4-8 8-4
4-8 0 459
8-4 466 0
All good.
In the matching task, we saved the names and face identity of the targets, based on what has been shown. It was also saved at the beginning of the task, after assigning those stimuli. Let’s test for consistency:
table(match.oName = allok$match.oName.y, oName = allok$oName, exclude = NULL)
oName
match.oName Andy Chris Daniel Jack James John Matt Michael Nick
Andy 118 0 0 0 0 0 0 0 0
Chris 0 105 0 0 0 0 0 0 0
Daniel 0 0 105 0 0 0 0 0 0
Jack 0 0 0 96 0 0 0 0 0
James 0 0 0 0 97 0 0 0 0
John 0 0 0 0 0 88 0 0 0
Matt 0 0 0 0 0 0 125 0 0
Michael 0 0 0 0 0 0 0 92 0
Nick 0 0 0 0 0 0 0 0 99
All seems fine.
table(match.yName = allok$match.yName.y, oName = allok$yName, exclude = NULL)
oName
match.yName Andy Chris Daniel Jack James John Matt Michael Nick
Andy 105 0 0 0 0 0 0 0 0
Chris 0 101 0 0 0 0 0 0 0
Daniel 0 0 104 0 0 0 0 0 0
Jack 0 0 0 112 0 0 0 0 0
James 0 0 0 0 102 0 0 0 0
John 0 0 0 0 0 99 0 0 0
Matt 0 0 0 0 0 0 103 0 0
Michael 0 0 0 0 0 0 0 95 0
Nick 0 0 0 0 0 0 0 0 104
All seems fine.
From the IAT, we can learn whether photos or names were presented.
table(IAT.stims = allok$iat.vCond, stimCond = allok$part3Cond, exclude = NULL)
stimCond
IAT.stims name-only name+photo
photo 0 351
word 574 0
All ok.
From the attribute questionnaire, we saved the Photo identity chosen (also saved in the matching task)
table(att.old.id = allok$att.man.id.o, match.old.id = allok$match.oFace.y, exclude = NULL)
match.old.id
att.old.id old_a old_b old_c old_d old_e old_f
old_a 140 0 0 0 0 0
old_b 0 162 0 0 0 0
old_c 0 0 133 0 0 0
old_d 0 0 0 154 0 0
old_e 0 0 0 0 170 0
old_f 0 0 0 0 0 166
table(att.yng.id = allok$att.man.id.y, match.yng.id = allok$match.yFace.y, exclude = NULL)
match.yng.id
att.yng.id young_a young_b young_c young_d young_e young_f
young_a 162 0 0 0 0 0
young_b 0 151 0 0 0 0
young_c 0 0 173 0 0 0
young_d 0 0 0 150 0 0
young_e 0 0 0 0 152 0
young_f 0 0 0 0 0 137
And the names:
table(att.old.name = allok$att.man.name.o, old.name = allok$oName, exclude = TRUE)
old.name
att.old.name Andy Chris Daniel Jack James John Matt Michael Nick
Andy 118 0 0 0 0 0 0 0 0
Chris 0 105 0 0 0 0 0 0 0
Daniel 0 0 105 0 0 0 0 0 0
Jack 0 0 0 96 0 0 0 0 0
James 0 0 0 0 97 0 0 0 0
John 0 0 0 0 0 88 0 0 0
Matt 0 0 0 0 0 0 125 0 0
Michael 0 0 0 0 0 0 0 92 0
Nick 0 0 0 0 0 0 0 0 99
table(att.yng.name = allok$att.man.name.y, yng.name = allok$yName, exclude = TRUE)
yng.name
att.yng.name Andy Chris Daniel Jack James John Matt Michael Nick
Andy 105 0 0 0 0 0 0 0 0
Chris 0 101 0 0 0 0 0 0 0
Daniel 0 0 104 0 0 0 0 0 0
Jack 0 0 0 112 0 0 0 0 0
James 0 0 0 0 102 0 0 0 0
John 0 0 0 0 0 99 0 0 0
Matt 0 0 0 0 0 0 103 0 0
Michael 0 0 0 0 0 0 0 95 0
Nick 0 0 0 0 0 0 0 0 104
Let’s also make sure all the name combinations occurred, but never using the same name for both targets:
table(old.name = allok$oName, yng.name = allok$yName, exclude = TRUE)
yng.name
old.name Andy Chris Daniel Jack James John Matt Michael Nick
Andy 0 18 18 11 15 13 13 16 14
Chris 14 0 12 14 14 13 12 13 13
Daniel 12 10 0 18 12 10 17 8 18
Jack 14 8 4 0 15 16 15 11 13
James 13 15 15 11 0 13 13 9 8
John 8 7 17 8 8 0 9 13 18
Matt 14 21 18 18 14 17 0 11 12
Michael 13 14 9 14 13 8 13 0 8
Nick 17 8 11 18 11 9 11 14 0
ok.
In the attributes questionnaire, we also saved the evaluation condition (with or without photos)
table(part3Cond = allok$part3Cond, attCond = allok$attPart3Cond, exclude = NULL)
attCond
part3Cond name-only name+photo
name-only 574 0
name+photo 0 351
ok ok.
One odd thing in the correlation matrix above was that the correlation between the IAT and self-reported preference changed when we correlated preference for the positively-portrayed target vs. when we correlated the preference for the young target. one was a recoding of the other (dependended on the induction condition), so it seems reasonable to expect the same correlation. To verify that no mistake was made, let’s see those correlations within each induction condition.
First, when the old target was the positively-portrayed target.
my.htmlTable(cornp(allok[which(allok$oldVlnc == "g"), c("IAT.gb", "IAT.prf", "eval.diff", "eval.gb")]))
| varName____ | IAT.gb____ | IAT.prf____ | eval.diff____ | |
|---|---|---|---|---|
| 1 | IAT.prf |
-1 < .001 466 |
||
| 2 | eval.diff |
-0.213 < .001 466 |
0.213 < .001 466 |
|
| 3 | eval.gb |
0.213 < .001 466 |
-0.213 < .001 466 |
-1 < .001 466 |
Second, when the old target was the negatively-portrayed target:
my.htmlTable(cornp(allok[which(allok$oldVlnc == "b"), c("IAT.gb", "IAT.prf", "eval.diff", "eval.gb")]))
| varName____ | IAT.gb____ | IAT.prf____ | eval.diff____ | |
|---|---|---|---|---|
| 1 | IAT.prf |
1 < .001 459 |
||
| 2 | eval.diff |
0.113 0.016 459 |
0.113 0.016 459 |
|
| 3 | eval.gb |
0.113 0.016 459 |
0.113 0.016 459 |
1 < .001 459 |
The important correlations were within the IAT scores and within the eval (self-report) scores. The -1 and 1 correlations show that the recoding was as expected. I am not entirely sure wha the reason for the difference in correlations is, when looking at the whole sample:
my.htmlTable(cornp(allok[, c("IAT.gb", "IAT.prf", "eval.diff", "eval.gb")]))
| varName____ | IAT.gb____ | IAT.prf____ | eval.diff____ | |
|---|---|---|---|---|
| 1 | IAT.prf |
-0.021 0.52 925 |
||
| 2 | eval.diff |
-0.008 0.816 925 |
0.237 < .001 925 |
|
| 3 | eval.gb |
0.171 < .001 925 |
-0.039 0.237 925 |
0.001 0.974 925 |
.237 vs. 0.171. Perhaps this is related to SD. Let’s verify that it doesn’t happen with Spearman correlation (i.e., the ranking stays the same)
my.htmlTable(cornp(allok[, c("IAT.gb", "IAT.prf", "eval.diff", "eval.gb")], type = "spearman"))
| varName____ | IAT.gb____ | IAT.prf____ | eval.diff____ | |
|---|---|---|---|---|
| 1 | IAT.prf |
0.002 0.962 925 |
||
| 2 | eval.diff |
0.025 0.45 925 |
0.233 < .001 925 |
|
| 3 | eval.gb |
0.164 < .001 925 |
-0.027 0.411 925 |
0.014 0.66 925 |
No. So it’s unrelated to SD.
Let’s simulate data and implement the same recoding, to make sure this can happen
df <- data.frame(sid = c(1:1000), one = rnorm(n = 1000, mean = 0.33, sd = 0.397), cond = rep(c(2, 1), 500))
df$two <- rnorm(n = nrow(df), mean = 3.779, sd = 53.916) + (df$one * 25)
df$one2 = ifelse(df$cond == 1, df$one, -1 * df$one)
df$two2 = ifelse(df$cond == 1, df$two, -1 * df$two)
my.htmlTable(cornp(df))
| varName____ | sid____ | one____ | cond____ | two____ | one2____ | |
|---|---|---|---|---|---|---|
| 1 | one |
-0.006 0.845 1000 |
||||
| 2 | cond |
-0.002 0.956 1000 |
-0.029 0.356 1000 |
|||
| 3 | two |
-0.039 0.214 1000 |
0.179 < .001 1000 |
0.026 0.414 1000 |
||
| 4 | one2 |
0.021 0.513 1000 |
0.031 0.334 1000 |
-0.634 < .001 1000 |
-0.029 0.364 1000 |
|
| 5 | two2 |
-0.042 0.181 1000 |
-0.011 0.739 1000 |
-0.177 < .001 1000 |
0.015 0.638 1000 |
0.249 < .001 1000 |
The comparison of interest is between the correlation of one with two and the correlation of one2 with two2. We can see that these are not the same correlations.
The age bias in the IAT was quite small, in comparison to previous studies.
in none of the 5 conditions, the age-bias was strong in both IAT block-order conditions. This is further evidence that we did not obtain strong and reliable age bias with the IAT.
There was some surprising age bias in the self-reported evaluation: in the [name-only,name-only] (d=0.33) and [name+age+photo,name-only] (d=0.49) conditions. Oddly, these conditions did not show age-bias in the IAT. It is difficult to understand what happened here, but might be worth replicating.
People had the best memory for behaviors attributed to the young target, followed by the old target, and then novel behaviors. That difference did not occur in the [name-only, name-only] baseline condition. People had more accurate memory, the more individuating information appeared in the induction task. We did not find evidence that individuating information in the memory test (retrieval) helped, but we also did not have conditions that provided those cues in both encoding and retrieval (not sure there is a reason to expect that cues at retrieval that did not appear in encoding would improve memory).